RECEIVED 

CENTRAL FAX CENTER 

Amendments to Claims: * ^ 

Please amend the claims as follows. This listing o f claims will replace all prior versions 
of the claims in the application. 



1, (Currently Amended) A computer-readable medium hayjng^ampriping software 
stored thereon for operating a video surveillance system, comprisin g cod e oegmonta for operating 
the video amveillonco sy s t e m basod on video primitives, wh e r e in the code oogmonfa for operating 
th e video surveillanc e syst e m compri se: 

(A) code segments for determining id e ntifying one or more user-defined event 
discriminator s, each user-defined event discriminator to detect an occurrence of a user-defined 
event in a video of the video surveillance system, each user-defined event comprising a ' 
description of at least one object engaged in an activity in the video, a description of at least one 
object eng aged in an activity having one or more spatial attributes in the video, a description of at 
least one o bject engaged in an activity having one or more temporal attributes in the video, or a 
description of at least o ne object engaged in an activity having one or more spatial attributes and 
one or more temporal attributes in the video, the object being an item of interest in the video : 

flBj code segments for extracting video primitives from the a video regardless of what or 
when eve nt discriminators are defined, each video primitive extracted being independent of any 
user-defined event described bv the user-defined event discriminators, wherein extracting video 
primitives comprises: 

U) identifying one or more objects in the video to obtain identified objects, each 
object b eing an item of i nterest in die video, wherein identifying one or more objects comprises 
at least one of: 
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fa) detecting one or more objects in the video; 

(b) tracking one or more objects in the video: or 

(c) . classifying one or more objects in the video; and 

(2) identifying at lea st one video primitive for each identified object in the video 
independent of anv user-defined event described by anv user-defined event discriminator, each 
video primitive descri bing a property of one of the identified objects, each property being an 
observable attribute of the identified object in the video : and 

fC) code seg ments for checking the extracted video primitives against at least one of the 
user-defined event discriminators to determine whether anv user-defined events described bv the 
checked user-defined event discriminators occurred in the video, wherein checking the extracted 
video Primitives comprises: 

( \) comparing the properties of the video primitives with the description of the 
user-defined event of one of the user-defined event discriminators: and » 

(2) determining a user-defined event occurred ,in_the video according to one of the 
user-defined event discriminators if the properties of the video primitives match the description 
of the user-defined event of one of the user-defmed event discriminators: 

wherein the code segments for extracting video primitives are different from the code 
segments for checking the extracted video primitives 

codo segments for extracting ov e nt occurr e nces fron - ndi e e xtract e d vide e^ prknitiv es using 
at l e ast - ono of th e on e or more uoor d e fin e d ev e nt difloriminators 3 wh e r e in tho oodo gogm e nts for 
extracting e v e nt oocuraenoes -a r -e^ diififer e nt from the oodo oogmonto for extracting vid e o primitiv e s; - 

whe r e i n each vid e o primitive is an obs e rvabl e attribute of an object viewed in tho video; 

w h e r e in th e video primitives ar e ot l e ast - o n e of the following: a siz e , a c hap e , a color, a 
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t e xture, a velocity, a apcod, on internal motion, a fcaturo of a salient motion, or a featu r e uf * 
sc e ne change . 



2. (Cancelled) 

3. (Cancelled) 

4. (Currently Amended) A computer-readable medium as in claim 1, further comprising: 
£D} code segments for archiving the extracted video primitives; 

wherein the arc hived video p r imitives are accessible without reprocessing the video . 

5. (Currently Amended) A computer-readable medium as in claim 1, further comprising 
code segments for undertaking a response based on the checked extracted video primitives 
e jctraotod -e v e nt ooourroncoG . 

6. (Original) A computer-readable medium as in claim 5, wherein the response 
comprises initiating another sensor system. 

7. (Original) A computer-readable medium as in claim I, further comprising code 
segments for calibrating the video surveillance system. 

8. (Original) A computer-readable medium as in claim 7 ? wherein the code segments for 
calibrating comprise code segments for self-calibrating the video surveillance system. 
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9. (Original) A computer-readable medium as in claim 8, wherein the code segments for 
self-calibrating comprise; 

code segments for detecting as least one object in a source video; and 
code segments for tracking the object. 

10. (Original) A computer-readable medium as in claim 9, wherein the code segments for 
detecting at least one object comprise: 

code segments for detecting at least one object via motion of the object; and 
code segments for detecting at least one object via change in a background model. 

1 1 . (Original) A computer-readable medium as in claim 7, wherein the code segments for 
self-calibrating comprise: 

code segments for identifying trackable areas; and 

code segments for identifying typical sizes of typical objects. 

12. (Original) A computer-readable medium as in claim 7, wherein the code segments for 
calibrating comprise; 

code segments for manual calibration; 

code segments for semi-automatic calibration; and 

code segments for automatic calibration, 

13. (Cancelled) 
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14. (Currently Amended) A computer-readable medium as in claim 1 1Z, wherein at 
least one user-defined event comprises a d escription of at least one object engaged in an activity 
in the video tho code aogmcnta for tooldng comprise oodo oogmonto for identifying at leapt quo 

1 5. (Currently Amended) A computer-readable medium as in claim I wherein gt 
least one user-defined e vent comprises a description of at least one object engaged in an acti vity 
having one or more sp a tial attributes inthe video & e» ood e segments for talking onmprir^ r,nrfr 
aogmcnta for identifying at least one s i patiol pro a. 

16. (Currently Amended) A computer-readable medium as in claim 1 4*, wherein at 
least one user-define d event comprises a description of at least one object engaged in an activity 
having one or more t e mporal attributes in the video th e cod e oegm o nta for tanking oornprine enda 
oogmonts for identifying at least on e temporal attrib ute. 

17. (Currently Amended) A computer-readable medium as in claim 1 whereinat 
least one user-defined event comprises a description of at least one object engaged in an activity 
having one or more sp atial attributes and one or more temporal attributes in the video fee-ee4e 
se gm e nts for tasking identify at boat ono int e raotioa 

18. (Currently Amended) A computer-readable medium as in claim I whereinat 
least one user-defined event comprises the code qogmento f or tasking identify at least one alarm. 
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1 9. (Currently Amended) A computer-readable medium as in claim 1 , wherein the 
video primitives are from at least one of a video sensor or aad another sensor. 

20. (Cancelled) 

21. (Original) A computer system comprising the computer-readable medium of claim 1 

22. (Cancelled) 

23. (Cancelled) 

24. (Currently Amended) A mgthod computer readable modium as in claim 25 £3, 
further comprising c e d e s e gm e n to for undertaking a response based on the checked extracted 
video Primitives e xtract e d evont nnnnrrnnnnn . 

25. (Currently Amended) A method comprising th e st e p of fgLoperating a video 
surveillance system, ba o od on video primitivca, wherein operating tho vidoo ourvoillanoc qyatam 
eempriseo tho steps o f comprising : 

£A) determining id e ntifying one or more user-defined event discriminators , each user- 
defined event discriminator to detect anoccurrence of a user-defined event in a video of the 
video surveillance system, each user-d efined event comprising a description of at least one object 
engaged in an activity in the video, a de scription of at least one object engaged in an activity 
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having one or morespatial attributes in the video, a description of at least one object engaged in 
an activity having one or mote temporal attri b utes in the video, or a description of at least one 
object engaged in an activity having one o r more spatial attributes and one or more temp oral 
attributes in the video, t he object being an item of interest in the video : 

(BLexttacting video primitives from the a video regardless of what or when evenr 
discriminators are defined, each video pri mitive extracted being independent of any user-defined 
event described by the user-defined e vent discriminators, wherein extracting video primitives 
comprises: 

m identifying one or more objects in the video to obtain identified objects, each 
object being an item of interest in the video, wherein identifying one or more objects comprises 
at least one of: 

(a) detecting one or more objects in the video; 

(b) tracking on e or more objects in the video: or 

(c) classifying one or more objects in the video: and 

(2^ iden tifying at least _one video primitive for each identified object in the video 
independent of anv u ser-defined event described bv any user-defined event discriminator, each 
video pri mitive describing a property of one of the identified objects, each property being an 
observable attribute of the identified object in the video : and 

(C) check ing the extracted video primitives_against at least one of the user-defined event 
discriminators to determine whether anv user-defined events described bv the checked user- 
defined ev ent discriminators occurred in the video, wherein checking the extracted video 
primitives comprises: 

(I) comparing the propertie s of the video primitives with the description of the 
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user-defined event of one of the user-defined event discriminators- and 

(2) determining a us<a>defined event occurred in the video according to one of dig 
user-defined event discriminators if the p rope rties of the video primitives match the descrip tion 
of the user-defined event of one of the user-defined, event discriminators: 

wherein extracting video pri mitives is separate from checking the extracted video 
primitives 

e xtracting event ooourrcnoc o fr om, the video primitives using at loaat one of the one or 
more user defined event discriminator; 

wh e rein oach vid e o primitive i ^ an obs e rvable attribute of on objoct viewed in - th e vide e? 

wherein tho video primitivca - cire at least one of the following: - a qiao, a shopp y a color, a 
te xture, a velocity, a apood> on internal motion, a feature of a salient motion, or a feature of a 
joono chongo . 

26* (Cancelled) 

27. (Currently Amended) A An apparatu s for - v ideo surveillance system, wherein tho 
apparatus is adapted to perform video ourvoiUanoe bas e d on vidoo primitivoo^ wherein tho 
apparatu s ! is adapted to comprising : 

(A) means for determining identifying one or more user-defined event discriminators^ 
each user-defined eve nt discriminator to detect an occurrence of a user-defined event in a video 
of the video surveillance system, eac h user-defined event comprising a description of at least one 
object engaged in an activity in the video, a description of at least one object engaged in an 
activity having one or m ore spatial attributes in the video, a description of at least one object 
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engaged to an activity havine one or more te mporal attributes in the video, or a descrip tion of «t 
least one object engaged in an activity having one or more spatial attributes and one or more 
temporal attributes in the viden 1 the nhjp rt being an item of interest in the video: 

,(B) means for cxtracti^ exfeae^video primitives from the a video regardless of what or 
when event discriminator s are defined, each video primitive extracted being independent of anv 
user-defined event described bv the us er-defined event discriminators, wherein the means for 
extracting video primitives comprises: 

(D means for identify i ng_oiie or more objects in the video to_obtain identified 
objects, each object bei ng an item of interest in the video, wherein the means for identifying one 
or more objects comprises at least one of: 

fa) means for detecting one or more objects in the video: 

(b) means for tracking one or more objects in the video; or 

(c) means for classifying one or more objects in the video: and 

(2) means for identifying at least one video primitive for each identified object in 
the video independent of anv user-defined event described bv anv user-defined event 
discriminator, each video primitive describing a property of one of the identified objects, each 
property being an observable attribute of the identified object in the video : and 

(C) means for checking the extracted video primiti ves against at least one of the user- 
defined event discriminators to determine whether any user-defined events described bv the 
checked use r-defined event discriminators occurred in the video, wherein third unit to check the 
extracted video primitives comprises: 

(I) means for comparing the properties of the video primitives with the 
descripti on of the user-defined event of one of the user-defined event discriminators: and 
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(2) means for determining a user- defined event occurred in the video according tn 
pnc of the user-defi ned event discriminato rs if the properties of the video primitives match tha 
d escription of the user-defined event of one of the user-defined event discriminatory 

wherein the means for extracti ng video primitives is different from the means for 
checking the extracted video primitives 

e xtract e vent oociuToncoo from th o vid e o primitivco using at least one of tho one or more 
user defined event dipcrimiaator^ 

wh e r e in each vid e o primitive ia on obs e rvable attribute of an object view e d in tho video* 

wheroin the video primitivca are at l e ast one of the following: a size, a shape, a oolor^ a 
texture, a velocity, a op ee d^ an internal motion, a feature of a aaliont motion, or a f e ature of a 
soono change . 

28. (Previously Presented) The apparatus of claim 27, wherein the apparatus comprises 
application-specific hardware to emulate a computer and/or software. 

29, (Currently Amended) A computer-readable medium as in claim 1 ? wherein at least 
one user-defined event discriminator is further checked against ooourroncon aro ejetractod boscd 
on video primitives and non-video primitives. 

30> (Currently Amended) A computer-readable medium as in claim 1, further 
comprising code segments for determining id e ntifying the one or more user-defined event 
discriminators using a user interface. 
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31. (Cancelled) 



32. (Cancelled) 

33. (Currently Amended) Xhe apparatus Ap pl ication n pocifio ha*kx^ gg \ n n*j m 9g 
wfreyein further comprising self calibrating the application-specific hardware is self- 
calibrating for performing video surveillance. 

34. (Currently Amended) A method Application - sp e cific hardware as in claim 25 32, 
wherein at least one user-defined event discriminator is further checked against ooouTroncofi n™ 
e xtracted bas e d on vid e o primitives ond non-video primitives. 

35. (Currently Amended) A method Application specific hardware as in claim 25 32, 
wherein at least one user-defined event discriminator comprises includes at leaat two of tho 
following: an object, a npatiol area, a lomporal - attributo, an interaction, or an at least one alarm. 

36. (Cancelled) 

37. (Previously Presented) A computer-readable medium as in claim 1, wherein the 
video primitives are at least two of the following: a size, a shape, a color, a texture, a position, a 
velocity, a speed, an internal motion, a feature of a salient motion, or a feature of a scene change. 

38. (Previously Presented) A computer-readable medium as in claim l s wherein the 
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video primitives are at least three of the following: a $ize, a shape, a color, a texture, a velocity, a 
speed, an internal motion, a feature of a salient motion, or a feature of a scene change. 

39. (Previously Presented) A computer-readable medium as in claim 1, wherein the 
video primitives are at least seven of the following: a classification, a size, a shape, a color, a 
texture, a position, a velocity, a speed, an internal motion, a motion, a salient motion, a feature of 
a salient motion, a scene change, a feature of a scene change, or a pre-defined model 

40. (Cancelled) 

41. (Currently Amended) A computer-readable medium as in claim 4 4Q, further 
comprising: 

code segments for determining i d e ntifying one or more additional user-defined event 
discriminators; and 

code segments for checking e xtracting w e nt occurenc e s from the archived video 
primitives against usieg at least one of the one or more additional user-defined event 
discriminators. 

42. (Currently Amended) A computer-readable medium as in claim 1 40, wherein the 
eaeh video primiti ves include primitive is-an obs e rvable attribute of an object viowod in the 
video and inoludoa at least one of the following: a size; a shape; a color; a texture; a position; a 
velocity; a speed; an internal motion; a feature of a salient motion; or a feature of a scene change. 
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43. (Currently Amended) A computer-readable medium haying_ pompri s ing -software 
stored thereon for operating a video surveillance system with user-defined event discrimintors . 
comprisin g code oogmcnto for operating th e video surveillance aystom based on video primitive^ 
wh e r e in th e code segmenta for operating the video purvoillanoc syst e m comprise : 

lALcode segments for extracting video primitives from a vide o regardless of what or 
when event discriminators are define d, each video primitive extracted being independent of any 
user-defined event described bv anv itser-defincd event discriminators, wherein extracting videq 
primitives comprises: 

^identifying one or more objects in the video to obtain identified objects, each 
object bei ng an item of interest in the video, wherein identifving^one or more objects comprises 
at least one of: 

fa) detecting one or more objects in the video: 

(b) tracking one or more objects in the video: or 

(c) __cla$sifving one or more objects in the video: and 

(2) identifying at least one vi deo primitive for each identified object in the vidcg 
independent of anv user- defined event described bv anv user-defined event discriminator, each 
video primi tive describing a property o f one o f the identified objects, each property being an 
observable attribute of the identified object in the video : and 

£B)_code segments for archiving the extracted video primitives; 

(O code segments for determining o ne or more user-defined event discriminators, each 
user-defin ed event discriminator to detect an occurrence of a user-defined event in the video of 
the video surveillan ce system, each user-defined event comprising a description of at least one 
object engaged in an activity in the video, a description of at least one object engaged in an 
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activity having one or more spatial attributes i n the video, a description of at least one object 
engaged in an activity having one or more temporal attributes in the video, or a description of at 
least one obj e ct e n gaged in an activity having one or more spatial attributes and one or more 
temporal attributes in the video, the nh fot being an item of interest in the video: 

(D) code segments for accessin g the archived video primitives corresponding to a portion 
of the video: and 

(B) code segments for checking th e accessed archived video primitives against at least 
one of the user-defined event discriminators to determine whether any user-defined events 
described by the checked user-defined event discriminators occurred in the video, wherein 
checking the extracted video primitives comprises: 

(1) compari ng the properties of the video primitives with the description of the 
user-defin ed event of one of the user-defined event discriminators: and 

(2) determinin g a user-define d event occurred in the video according to one of the 
user-defined event disc riminators if the properties of the video primitives match the description 
of the user-defined eve nt of one of the user-defined event discriminators: 

wherein th e archived video p ri mitives are accessed without reprocessing the video; 

wherein the code segments for extracting video primitives are different from the code 
segments for checking the accessed archived video primitives; 

wherein the accessed archived video primitives are checked against at least one of the 
user-defined event discriminators without reprocessing the portion of the video . 

44. (Currently Amended) A computer-readable medium as in claim 43, wherein the sash 
video primitives include primitive is cu^bs e rvable attribut e of an object viewed in the video mid 
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ieeiudes at least one of the following: a size; a shape; a color; a texture; a position; a velocity; a 
speed; an internal motion; a feature of a salient motion; or a feature of a scene change. 

45. (New) A method as in claim 25, further comprising: 
(D) archiving the extracted video primitives; 

wherein the archived video primitives are accessible without reprocessing the video. 

46. (New) A method as in claim 25, wherein at least one user-defined event comprises a 
description of at least one object engaged in an activity in the video. 

47. (New) A method as in claim 25, wherein at least one user-defined event comprises a 
description of at least one object engaged in an activity having one or more spatial attributes in 
the video. 

48. (New) A method as in claim 25, wherein at least one user-defined event comprises a 
description of at least one object engaged in an activity having one or more temporal attributes in 
the video. 

49. (New) A method as in claim 25, wherein at least one user-defined event comprises a 
description of at least one object engaged in an activity having one or more spatial attributes and 
one or more temporal attributes in the video. 

50. (New) A video surveillance system as in claim 27, further comprising: 
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(D) means for archiving the extracted video primitives; 

wherein the archived video primitives are accessible without reprocessing the video. 

5 1 . (New) A method for operating a video surveillance system with user-defined event 
discriminators* comprising; 

(A) extracting video primitives from a video regardless of what or when event 
discriminators are defined, each video primitive extracted being independent of any user-defined 
event described by any user-defined event discriminators, wherein extracting video primitives 
comprises: 

(1) identifying one or more objects in the video to obtain identified objects, each 
object being an item of interest in the video, wherein identifying one or more objects comprises 
at least one of: 

(a) detecting one or more objects in the video; 

(b) tracking one or more objects in the video; or 

(c) classifying one or more objects in the video; and 

(2) identifying at least one video primitive for each identified object in the video 
independent of any user-defined event described by any user-defined event discriminator, each 
video primitive describing a property of one of the identified objects, each property being an 
observable attribute of the identified object in the video; and 

(B) archiving the extracted video primitives; 

(C) determining one or more user-defined event discriminators, each user-defined event 
discriminator to detect an occurrence of a user-defined event in the video of the video 
surveillance system, each user-defined event comprising a description of at least one object 
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engaged in au activity in the video, a description of at least one object engaged in an activity 
having one or more spatial attributes in the vi deo, a description of at least one object engaged in 
an activity having one or more temporal attributes in the video, or a description of at least one 
object engaged in an activity having one or more spatial attributes and one or mote temporal 
attributes in the video, the object being an item of interest in the video; 

(D) accessing the archived video primitives corresponding to a portion of the video; and 

(E) checking the accessed archived video primitives against at least one of the user- 
defined event discriminators to determine whether any user-defined events described by the 
checked user-defined event discriminators occurred in the video, wherein checking the extracted 
video primitives comprises: 

(1) comparing the properties of the video primitives with the description of the 
user-defined event of one of the user-defined event discriminators; and 

(2) determining a user-defined event occurred in the video according to one of the 
user-defined event discriminators if the properties of the video primitives match the description 
of the user-defined event of one of the user-defined event discriminators; 

wherein the archived video primitives are accessed without reprocessing the video; 
wherein extracting video primitives is different from checking the accessed archived 
video primitives; 

wherein the accessed archived video primitives are checked against at least one of the 
user-defined event discriminators without reprocessing the portion of the video. 

52. (New) A video surveillance system employing user-defined event discriminators, 
comprising: 
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(A) means for extracting video primitives from a video regardless of what or when event 
discriminators are defined, each video primitive extracted being independent of any user-defined 
event described by any user-defined event discriminators, wherein the means for extracting video 
primitives comprises; 

(1) means for identifying one or more objects in the video to obtain identified 
objects, each object being an item of interest in the video, wherein the means for identifying one 
or more objects comprises at least one of: 

(a) means for detecting one or more objects in the video; 

(b) means for tracking one or more objects in the video; or 

(c) meaus for classifying one or more objects in the video; and 

(2) means for identifying at least one video primitive for each identified object in 
the video independent of any user-defined event described by any user-defined event 
discriminator, each video primitive describing a property of one of the identified objects, each 
property being an observable attribute of the identified object in the video; and 

(B) means for archiving the extracted video primitives; 

(C) means for deteonining one or more user-defined event discriminators, each user- 
defined event discriminator to detect an occurrence of a user-defined event in the video of the 
video surveillance system, each user-defined event comprising a description of at least one object 
engaged in an activity in the video, a description of at least one object engaged in an activity 
having one or more spatial attributes in the video, a description of at least one object engaged in 
an activity having one or more temporal attributes in the video, or a description of at least one 
object engaged in an activity having one or more spatial attributes and one or more temporal 
attributes in the video, the object being an item of interest in the video; 
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(D) means for accessing the archived video primitives corresponding to a portion of the 
video; and 

(E) means for checking the accessed archived video primitives against at least one of the 
user-defined event discriminators to determine whether any user-defined events described by the 
checked user-defined event discriminators occurred in the video, wherein the means for checking 

, the extracted video primitives comprises: 

(1) means for comparing the properti es of the video primitives with the 
description of the user-defined event of one of the user-defined event discriminators; and 

(2) means for determining a user-defined event occurred in the video according to 
one of the user-defined event discriminators if the properties of the video primitives match the 
description of the user-defined event of one of the user-defined event discriminators; 

wherein the archived video primitives are accessed without reprocessing the video; 

wherein the means for extracting video primitives is different from the means for 
checking the accessed archived video primiti ves; 

wherein the accessed archived video primitives are checked against at least one of the 
user-defined event discriminators without reprocessing the portion of the video. 

53. (New) A computer-readable medium having software stored thereon for operating a 
video surveillance system, comprising: 

(A) code segments for determining one or more first user-defined event discriminator 
each first user-defined event discriminator to detect an occurence of a user-defined event in a 
video of the video surveillance system, each user-defined event comprising a description of at 
least one object engaged in an activity in the video, a description of at least one object engaged in 
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an activity having one or mote spatial attributes in the video, a description of at least one object 
engaged in an activity having one or more temporal attributes in the video, or a description of at 
least one object engaged in an activity having one or more spatial attributes and one or more 
temporal attributes in the video, the object being an item of interest in the video; 

(B) code segments for extracting video primitives from the video regardless of what or 
when event discriminators are defined, each video primitive extracted being independent of any 
user-defined event described by any u$er-defined event discriminators, wherein extracting video 
primitives comprises: 

(1 ) identifying one or more objects in the video to obtain identified objects, each 
object being an item of interest in the video, wherein identifying one or more objects comprises 
at least one of: 

(a) detecting one or more objects in the video; 

(b) tracking one or more objects in the video; or 

(c) classifying one or more objects in the video; and 

(2) identifying at lea$t one video primitive for each identified object in the video 
independent of any user-defined event described by any user-defined event discriminator,, each 
video primitive describing a property of one of the identified objects, each property being an 
observable attribute of the identified object in the video; 

(C) code segments for archiving the extracted video primitives; 

(D) code segments for checking the extracted video primitives against at least one of the 
first user-defined event discriminators to determine whether any user-defined events described by 
the checked first user-defined event discriminators occurred in the video, wherein checking the 
extracted video primitives comprises: 
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(1) comparing the properties of the video primitives with the description of the 
user-defined event of one of the first user-defined event discriminators; and 

(2) determining a user-defined event occurred in the video according to one of the 
first user-defined event discriminators if the properties of the video primitives match the 
description of the user-defined event of one of the first user-defined event discriminators; 

(E) code segments for determining one or more second user-defined event 
discriminators, each second user-defined event discriminator to detect an occurrence of a user- 
defined event in the video of the video surveillance system, each user-defined event comprising a 
description of at least one object engaged in an activity in the video, a description of at least one 
object engaged in an activity having one or more spatial attributes in the video, a description of at 
least one object engaged in an activity having one or more temporal attributes in the video, or a 
description of at least one object engaged in an activity having one or more spatial attributes and 
one or more temporal attri butes in the video, the object being an item of interest in the video; 

(F) code segments for accessing the archived video primitives corresponding to a portion 
of the video; 

(G) code segments for checking the accessed video primitives against at least one of the 
second user-defined event discriminators to determine whether any user-defined events described 
by the checked second user-defined event discriminators occurred in the video, wherein checking 
the accessed video primitives comprises: 

(1) comparing the properties of the accessed video primitives with the description 
of the user-defined event of one of the second user-defined event discriminators; and 

(2) determining a user-defined event occurred in the video according to one of the 
second user-defined event discriminators if the properties of the accessed video primitives match 
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the description of the userniefined event of one of the second user-defined event discriminators; 
wherein the code segments for extracting video primitives are different from the code 
. segments for checking the extracted video primitives; 

wherein the accessed video primitives are checked against at least one of the second user- 
defined event discriminators without reprocessing the portion of the video. 

54. (New) A method for operating a video surveillance system, comprising: 

(A) determining one or more first user-defined event discriminators* each first user- 
defined event discriminator to detect an occurrence of a user-defined event i n a video of the 
video surveillance system, each user-defined event comprising a description of at least one object 
engaged in an activity in the Video, a description of at least one object engaged in an activity 
having one or more spatial attributes in the video, a description of at least one object engaged in 
an activity having one or more temporal attributes in the video, ox a description of at least one 
object engaged in an activity having one or more spatial attributes and one or more temporal 
attributes in the video, the object being an item of interest in the video; 

(B) extracting video primitives from the video regardless of what or when event 
discriminators are defined, each video primitive extracted being independent of any user-defined 
event described by any user-defined event discriminators, wherein extracting video primitives 
comprises; 

(1) identifying one or more objects in the video to obtain identified objects, each 
object being an item of interest in the video, wherein identifying one or more objects comprises 
at least one of: 

(a) detecting one or more objects in the video; 
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(b) tracking one or more objects in the video; or 

(c) classifying one or more objects in the video; and 

(2) identifying at least one video primitive for each identified object in the video 
independent of any user-dej&ned event described by any usei^defined event discriminator, each 
video primitive describing a property of one of the identified objects, each property being an 
observable attribute of the identified object in the video; 

(C) archiving the extracted video primitives; 

(D) checking the extracted video primiti ves against at least one of the first user-defined 
event discriminators to determine whether any user-defined events described by the checked first 
user-defined event discriminators occurred in the video, wherein checking the extracted video 
primitives comprises: 

(1) comparing the properties of the video primitives with the description of the 
user-defined event of one of the first user-defined event discriminators; and 

(2) determining a user-defined event occurred in the video according to one of the 
first user-defined event discriminators if the properties of the video primitives match the 
description of the user-defined event of one of the first user-defined event discriminators; 

(E) determining one or more second user-defined event discriminators, each second user- 
defined event discriminator to detect an occurrence of a user-defined event in the video of the 
video surveillance system, each user-defined event comprising a description of at least one object 
engaged in an. activity in the video, a description of at least one object engaged in an activity 
having one or more spatial attributes In the video, a description of at least one object engaged in 
an activity having one or more temporal attributes in the video, or a description of at least one 
object engaged in an activity having one or more spatial attributes and one or more temporal 

- 24 - Application No. 09/987,707 

PAGE 26/37 * RCVD AT 9/19/2007 5:46:57 PM [Eastern Daylight Time) * SVR:USPTO-EFXRF-3/4 * DNIS:2738300 * CSID: ' DURATION (mm-ss):09-54 



attributes in the video, the object being an item of interest in the video; 

(F) accessing the archived video primitives corresponding to a portion of the video; 

(G) checking the accessed video primitives against at least one of the second user-defined 
event discriminators to determine whether any user-defined events described by the checked 
second user-defined event discriminators occurred in the video, wherein checking the accessed 
video primitives comprises: 

(1) comparing the properties of the accessed video primitives with the description 
of the user-defined event of one of the second user-defined event discriminators; and 

(2) determining a user-defined event occurred in the video according to one of the 
second user-defined event discriminators if the properties of the accessed video primiti ves match 
the description of the user-defined event of one of the second user-defined event discriminators; 

wherein extracting video primitives is different from checking the extracted video 
primitives; 

wherein the accessed video primitives are checked against at least one of the second user- 
defined event discriminators without reprocessing the portion of the video. 

55. (New) A video surveillance system, comprising: 

(A) means for determining one or more first user-defined event discriminators, each first 
user-defined event discriminator to detect an occurrence of a user-defined event in a video of the 
video surveillance system, each user-defined event comprising a description of at least one object 
engaged in an activity in the video, a description of at least one object engaged in an activity 
having one or more spatial attributes in the video, a description of at least one object engaged in 
an activity having one or more temporal attributes in the video, or a description of at least one 
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object engaged in an activity having one or more spatial attributes and one or more temporal 
attributes in the video, the object being an item of interest in the video; 

(B) means for extracting video primitives from the video regardless of what or when 
event discriminators are defined, each video primitive extracted being independent of any user- 
defined event described by any user-defined event discriminators, wherein the means for 
extracting video primitives comprises: 

(1) means for identifying one or more objects in the video to obtain identified 
objects, each object being an item of interest in the video, wherein the means for identifying one 
or more objects comprises at least one of: 

(a) means for detecting one or more objects in the video; 

(b) means for tracking one or more objects in the video; or 

(c) means for classifying one or more objects in the video; and 

(2) means for identifying at least one video primitive for each identified object in 
the video independent of any user-defined event described by any user-defined event 
discriminator, each video primitive describing a property of one of the identified objects, each 
property being an observable attribute of the identified object in the video; 

(C) means for archiving the extracted video primitives; 

(D) means for checking the extracted video primitives against at least one of the first 
user-defined event discriminators to determine whether any user-defined events described by the 
checked first user-defined event discriminators occurred in the video, wherein the means for 
checking the extracted video primitives comprises: 

(1) means for comparing the properties of the video primitives with the 
description of the user-defined event of one of the first user-defined event discriminators; and 
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(2) means for determining a user-defined event occurred in the video according to 
one of the first user-defined event discriminators if the properties of the video primitives match 
the description of the user-defined event of one of the fixst user-de6ned event discriminators; 

(E) means for determining one or more second user-defined event discriminators, each 
second user-defined event discriminator to detect an occurrence of a user-defined event in the 
video of the video surveillance system, each user-defined event comprising a description of at 
least one object engaged in an activity in the video, a description of at least one object engaged in 
an activity having one or more spatial attributes in the video, a description of at least one object 
engaged in an activity having one or more temporal attributes in die video, or a description of at 
least one object engaged in an activity having one or more spatial attributes and one or more 
temporal attributes in the video, the object being an item of interest in the video; 

(F) means for accessing the archived video primitives corresponding to a portion of the 

video; 

(G) means for checking the accessed video primitives against at least one of the second 
user-defined event discriminators to determine whether any user-defined events described by the 
checked second user-defined event di$criminators occurred in the video, wherein the means for 
checking the accessed video primitives comprises: 

(1) means for comparing the properties of the accessed video primitives with the 
description of the user-defined event of one of the second user-defined event discriminators; and 

(2) means for determining a user-defined event occurred in the video according to 
one of the second user-defined event discriminators if the properties of the accessed video 
primitives match the description of the user-defined event of one of the second user-defined 
event discriminators; 
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wherein the means for extracting video primitives is different from the means for 
checking the extracted video primitives; 

wherein the accessed video primitives are checked against at least one of the second user- 
defined event discriminators without reprocessing the portion of the video. 
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